Repository navigation
Add experimental Vulkan GPU rendering and export - #1138
Merged
Merged
Conversation
Work-in-progress Vulkan MVP, built only with -DENABLE_GPU_FRAME=ON. CPU builds are unchanged. - Frames can hold an immutable deferred GPU image and only read back CPU pixels when GetImage() is called. - FFmpegReader decodes through a shared Vulkan device and scales on the GPU. - Clip transforms and source-over compositing run in a Vulkan compute shader. - FFmpegWriter hands GPU frames to NVENC through Vulkan/CUDA interop. - VulkanOpenGLPresenter draws GPU frames in a QOpenGLWidget without readback. - Timeline and export benchmarks compare CPU and Vulkan builds.
Compile the compute shaders into C headers at build time instead of loading SPIR-V from absolute build-directory paths, so installed and packaged GPU builds work.
- OPENSHOT_RENDER_BACKEND=cpu now disables Vulkan decoding at runtime in a GPU build, so every frame follows the unchanged CPU path. Output is identical to a CPU build. - A failed GPU export no longer aborts. The writer reads the frame back and encodes the rest of the export from CPU pixels. - Compare clip composite modes against openshot::CompositeType.
Vulkan tests no longer need opt-in environment variables. They detect a Vulkan device, display and OpenGL interop themselves and print a "Skipping Vulkan test:" message that CTest reports as skipped, not passed. The reader cache test uses bundled media by default, and the renderer test waits for queued delivery on any Qt platform.
Each pipeline now reuses a small ring of command buffers, fences and descriptor sets. A submit no longer blocks the CPU; later readers wait on the frames' timeline semaphores, and each slot keeps its frames alive until the GPU is done with them.
Timeline layers only need the composited result, so draw the transformed clip straight onto the background. Layers with after-keyframe effects or timeline effects keep the separate passes.
Freed GPU frames return to FFmpeg pools that never shrink. Keep the compositor pools in one shared list, so ReleaseVulkanMemory() can drop them; FFmpeg frees each pool once its last frame is released. Submit slots no longer hold frame references, since FFmpeg waits on a frame's semaphores before freeing it.
A decoded frame was held until the audio stream was a full second past it, so every reader kept 1-2 seconds of decoded images. With GPU decoding that cost about 0.9 GB of video memory per 4K clip. A 0.25 second margin gives identical playback audio on 22 test files (AAC, MP3, Vorbis, Opus, FLAC, AC3, ALAC and PCM) and cuts that to about 0.34 GB.
FFmpeg binds a Vulkan frame to the first CUDA context it is sent to, and pooled frames were reused by the next export. The second export in a session then fell back to the CPU, and the third could crash. Release the pools when a writer opens and closes, and report export and decode fallbacks as warnings so they show in the console.
FFmpeg 7's MP4 reader returns 0,0,0 orientation, like 8 and 9. The written angles are still checked directly in the file.
The runner image now has FFmpeg 7.1.3 with Vulkan and newer Vulkan headers in /opt. The job fails if the system FFmpeg 4.2 is also linked.
FFmpeg uses the first Vulkan device, which is often a software driver such as llvmpipe, so decoding quietly fell back to the CPU. Try GPUs by name, discrete first, skip software devices, and log the chosen device. OPENSHOT_VULKAN_DEVICE overrides the choice.
AMD and Intel exports no longer copy each frame through system memory. The final NV12 images use linear DMA-BUF memory that VA-API imports directly, and the VA-API encoder opens on the card that renders. If sharing fails, the export continues with the old copy path. New helpers for the UI: ListGpuDevices(), ActiveGpuDevice() and GpuEncoders(). The Vulkan tests now use the same device choice as the app, so they no longer skip when a software driver is listed first.
Write native Vulkan NV12 directly into encoder-owned frames, with CPU fallback for unavailable encoders and incompatible surfaces. Keep encoder state per writer and preserve NVENC's separate-plane CUDA transfer. Add copy-free and fallback regression tests, readback diagnostics, and GPU compilation in the FFmpeg compatibility matrix. Verified on the Radeon 780M with FFmpeg 6.1 and 7.1.3; NVIDIA hardware testing remains outstanding.
Ubuntu 20.04's loader (1.2.131) hands FFmpeg a stub without a driver function for the video encode queries, and opening h264_vulkan crashes (FFmpeg's own CLI crashes the same way). Probe the native Vulkan encoders only with loader 1.3.270 or newer, the oldest verified to fail cleanly. Use h264_vulkan/hevc_vulkan only when the probe opened them, falling back to the CPU encoder otherwise, instead of trusting that FFmpeg includes them. Verified on an RTX 5070 (driver 580) with loaders 1.2.131 and 1.3.270: Vulkan tests pass, NVENC exports stay on the GPU, and the Vulkan presets fall back to libx264/libx265.
Keep playback and export counters separate, attribute CPU readbacks, and preserve measurement resets across in-flight work. Cache bounded GPU uploads for image sources so scaling, compositing, and presentation can stay on GPU.
Normalize uploaded artwork into the IOSurface compositor path so CPU fallback uses the synchronized Metal readback bridge. Assert the Mac surface type and retain strict pixel checks with useful RGBA diagnostics.
NVIDIA's Windows driver returns zeros when FFmpeg or compositor frames are read through an exported alias of their memory, from OpenGL or Vulkan, so native preview drew black. Copy each frame on the GPU into an exportable image owned by the presenter, which OpenGL imports once per size, device, and context, along with the semaphores, shader, and vertex state. The frame no longer changes queue ownership, which removes the invalid external acquire on concurrent FFmpeg images. The previous frame's fence is waited at the next draw instead of finishing GL every frame.
…wn at exit Maxwell GPUs raise a TEX LAYOUT exception, then a driver reset, when OpenGL samples an imported optimal-tiling Vulkan image. Copy each frame into an exported buffer instead and let OpenGL unpack it into its own texture. The shared Vulkan device and compositor output pools are no longer freed by static destructors, which ran during DLL unload after the NVIDIA driver had begun shutting down.
Its static destructor freed cached Vulkan frames during DLL unload, after the NVIDIA driver had begun shutting down, crashing OpenShot on exit.
Reuse one Vulkan frame pool per image size; a pool per upload leaked device memory. Share still and cover-art images between frames instead of deep copying them, so they are uploaded once instead of every frame. Wait for FFmpeg's upload to finish before freeing the CPU pixels it may read directly through host-mapped memory. Name the VkResult in presenter and compositor errors, count failed GPU uploads (logged every 100th) and stop reporting CPU-decoded cover art as missing GPU decode.
…ice loss A failed GPU preview draw now frees cached GPU memory and retries after a growing pause (0.5 s doubling to 30 s) instead of failing on every frame. VK_ERROR_DEVICE_LOST marks the device lost for the session: Vulkan rendering is disabled and open readers reopen with CPU decoding, so editing continues without a restart.
OPENSHOT_GPU_DEBUG=1 logs centre pixels at each preview handoff step, the shared buffer setup, OpenGL import errors, and draw geometry, and saves a crash dump on Windows. OPENSHOT_PRESENTATION_TEST=pattern, OPENSHOT_PRESENTATION_SYNC=wait and OPENSHOT_PRESENTATION_DEDICATED=0|1 change one part of the handoff each. Vulkan readback failures now log the frame and device details. Search for GpuDebug to remove.
The RTX 2060 debug logs showed OpenGL reading zeros from the shared preview buffer while Vulkan wrote the right pixels, even with a fill pattern and CPU waits. The presenter closed each Win32 handle right after importing it; keep them open until the shared target is released. OPENSHOT_PRESENTATION_HANDLE=kmt tries global KMT handles instead. With OPENSHOT_GPU_DEBUG=1, readbacks log their reason, and thumbnails and image saves are tagged.
The NVENC converter's ring slots kept their last frames, which were mapped into CUDA, until process exit. By then CUDA had unloaded, so FFmpeg logged dozens of failed CUDA cleanup calls after the session ended.
Select FFmpeg's native AV1 decoder on Vulkan devices advertising AV1 decode support, preserving the default decoder for software retries. Add hardware decode and output checks. Validation: three targeted AV1 tests pass on AMD Radeon 780M with Vulkan video enabled. Packaged FFmpeg builds including libdav1d still need validation.
Keep common image effects and scope analysis on GPU surfaces, reuse pipelines across preview sizes, and retain per-submission color grading parameters. Restore Qt texture bindings after native presentation. Add coverage for animated effects, image sizing, pipeline reuse, GPU residency, and presentation state. Native presentation passes 219 assertions; broader compositor and CPU checks passed during development. A device-loss failure during GUI seek/edit/layout smoke testing remains unresolved.
Patch FFmpeg 7.1 dependency builds to use staging memory for AMD Linux CPU transfers and correct the staging download copy direction. Preserve GPU-to-GPU video paths. Detect device loss hidden by FFmpeg transfer errors and prevent preview fallback from reading dead GPU frames. Verify the original reproducer, GUI smoke sequence, 906 compositor assertions, and 222 presentation assertions.
Guard the reader's device-loss check and the writer's Vulkan encoder check so builds without GPU frames compile against FFmpeg versions that predate AV_PIX_FMT_VULKAN.
Give ColorMap a Vulkan path, which ColorGrade's LUT now reuses, so LUTs no longer send grading back to the CPU. Blend effect masks on the GPU for effects that render on Vulkan, and count FrameScope histograms, waveforms and the vectorscope on the GPU, reading back only the counters. Masks match the CPU byte for byte and scope counts match exactly; LUTs stay within one byte. Failures fall back to the CPU. Add GPU-versus-CPU coverage for LUTs, masks and scopes, plus a masked, LUT-graded clip with scopes that needs no readbacks.
Blend Blur and Saturation drive-amount masks on the GPU with the same exact factor tables as the default blend. Read video masks from the decoder's GPU frames and convert and scale them in one compositor pass, for both the Mask effect and common effect masks. Still masks keep their cached CPU scaling and single upload. Add coverage for drive-amount masks against the CPU and for 720p video masks with no frame readbacks.
MoltenVK compiles shaders with fast math, which may fuse the mask blend's multiply-add and round one byte differently. Linux and Windows keep the byte-exact check.
- Decode with the source's tagged YUV matrix (BT.601 when untagged, as earlier OpenShot exports were written) on CPU and Vulkan. - Convert HLG and PQ (BT.2020) sources to SDR BT.709 with a shared, hue-preserving tone curve (BT.2408 reference white at 203 nits). CPU uses a LUT-based converter; Vulkan converts in affine.comp. - Read P010/P012/P016 and P210/P212/P216 frames directly on Vulkan, including 10-bit VideoToolbox surfaces on macOS. - Tag exports with BT.709 (HD) or BT.601 (SD) and encode with the matching matrix, on CPU and Vulkan. - Add SDR/HLG/PQ color chart fixtures and tests comparing CPU, Vulkan and expected values.
- Set OPENSHOT_RENDER_BACKEND with qputenv/qunsetenv; MinGW has no setenv/unsetenv. - Skip charts the hardware decoder declines on every platform. VideoToolbox on Intel Macs rejects the 320x192 H.264 chart; the Mac 10-bit surface path is covered by the synthetic CVPixelBuffer tests.
VideoToolbox on an Intel Mac (UHD 630) refuses 320x192 H.264 with "VideoToolbox session not available" (FFmpeg requires its hardware decoder for H.264), while 640x360 and larger decode. Regenerate the charts at 640x384 and require GPU decoding of them on macOS again.
libopenshot no longer links libvulkan on Linux or Windows. It opens the Vulkan library on first use and loads only the functions it calls. When the library or a function is missing (headless servers, old systems without a GPU driver), OpenShot still starts and renders on the CPU. macOS keeps linking the bundled MoltenVK.
Member
Author
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #1138 +/- ##
===========================================
+ Coverage 74.83% 75.22% +0.39%
===========================================
Files 276 282 +6
Lines 38296 39090 +794
===========================================
+ Hits 28659 29407 +748
- Misses 9637 9683 +46 ☔ View full report in Codecov by Harness. |
Audio-only readers sized their frame cache from video dimensions, so the cache never evicted and every insert rescanned every cached frame. Long waveform reads became quadratic: 100 s for a 2 hour file, now 6 s. - Size audio-only FFmpegReader caches by audio, and keep the cache of readers owned by AudioWaveformer tiny (reads are strictly sequential) - Add ExtractChannelSamples() to decode the mixed envelope and every channel's envelope in a single pass - Place chunk boundaries exactly, so 44.1 kHz audio no longer drifts (16 s late after 2 hours) or loses the end of the file - Add an ApplyKeyframes() overload taking plain envelopes, interpolating keyframes within a frame and fixing a one frame offset
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds an experimental GPU pipeline for video decoding, editing, preview, and export. Supported operations keep frames on the GPU, while other operations continue through the CPU path.
.cubeLUTs, including animated controls.Support limits: GPU acceleration depends on the operating system, device, driver, codec, and available sharing features. Other effects and unsupported rendering paths use CPU fallback; GPU output is not guaranteed to be pixel-identical to CPU output. Native Vulkan H.264/HEVC encoding requires a newer FFmpeg build than the packaged FFmpeg 7.1.
Companion application PR: OpenShot/openshot-qt#6242.